DOMAIN: Automotive Surveillance.

CONTEXT:Computer vision can be used to automate supervision and generate action appropriate action trigger if the event is predicted from the image of interest. For example a car moving on the road can be easily identified by a camera as make of the car, type, colour, number plates etc.

DATA DESCRIPTION:The Cars dataset contains 16,185 images of 196 classes of cars. The data is split into 8,144 training images and 8,041 testing images, where each class has been split roughly in a 50-50 split. Classes are typically at the level of Make, Model, Year, e.g. 2012 Tesla Model S or 2012 BMW M3 coupe.

Loading Libraries

In [1]:
from sklearn.preprocessing import StandardScaler # StandardScaler is used to resize the distribution of values,
# so that the mean of the observed values is 0 and the standard deviation is 1

import matplotlib.pyplot as plt # Matplotlib is a cross-platform, data visualization and graphical plotting library

import numpy as np # NumPy is a Python library used for working with arrays & linear algebra

import os # The OS module in Python provides functions for creating and removing a directory (folder),
# fetching its contents, changing and identifying the current directory, etc.

import pandas as pd # data processing, in this project used for CSV file I/O (e.g. pd.read_csv)

Importing and Exploring Supporting Data

Mounting Google Drive, Data has been downloaded and stored in Google Drive. Using which user can access the supporting files like Annotations, Car Images and Car Names

In [2]:
from google.colab import drive
drive.mount('/content/drive')
Drive already mounted at /content/drive; to attempt to forcibly remount, call drive.mount("/content/drive", force_remount=True).

Listing the Files and Folders available under Capstone Project

In [42]:
print(os.listdir('/content/drive/MyDrive/CapstoneProject/'))
# listdir() method in python is used to get the list of all files and directories in the specified directory
['CarNames.csv', 'Annotations', 'CarImages', 'CarImages_Cropped']

User Defined Function to create Correlation Plot Matrix

In [43]:
# User-defined function for "Correlation matrix"

def plotCorrelationMatrix(df, graphWidth): # 'df'-dataframe; 'graphwidth'-width (units = inches) are the inputs to this function

    """User Defined function to create Correlation matrix"""
    
    filename = df.dataframeName # the dataframe 'df' name is assigned to 'filename' variable
    
    df = df.dropna('columns') # drop columns with 'NaN'
    
    df = df[[col for col in df if df[col].nunique() > 1]] # keep columns where there are more than 1 unique values
    
    if df.shape[1] < 2: # checking if the dataframe has only one column, in such case no correlation can takes place
        print(f'No correlation plots shown: The number of non-NaN or constant columns ({df.shape[1]}) is less than 2')
        return
    
    corr = df.corr() # corr() is used to find the pairwise correlation of all columns in the dataframe
    
    plt.figure(num=None, figsize=(graphWidth, graphWidth), dpi=80, facecolor='w', edgecolor='k')
    # Create a new figure, or activate an existing figure
    # num = A unique identifier for the figure. 'None' = a new figure is created
    # figsize = Width, height in inches. Can be (float, float)
    # dpi = The resolution of the figure in dots-per-inch.
    # facecolor = The background color.
    # edgecolor = The border color.
    
    corrMat = plt.matshow(corr, fignum = 1)
    # matshow = Display an array as a matrix in a new figure window.
    # The origin is set at the upper left hand corner and rows (first dimension of the array) are displayed horizontally.
    # corr = The matrix to be displayed.
    # fignum = If None, create a new figure window with automatic numbering.
    # fignum = If a nonzero integer, draw into the figure with the given number (create it if it does not exist).
    # fignum = If 0, use the current axes (or create one if it does not exist).
    
    plt.xticks(range(len(corr.columns)), corr.columns, rotation=90)
    # xticks = Get or set the current tick locations and labels of the x-axis
    # range(len()) = to access each item by index in columns of corr
    # corr.columns = The list of xlabel Text objects
    # rotation = rotating xlabels
    
    plt.yticks(range(len(corr.columns)), corr.columns)
    # similar to above 'xticks', but with 'y'
    
    plt.gca().xaxis.tick_bottom()
    # 
    
    plt.colorbar(corrMat)
    plt.title(f'Correlation Matrix for {filename}', fontsize=15)
    plt.show()
In [44]:
print(plotCorrelationMatrix.__doc__)
User Defined function to create Correlation matrix
In [45]:
# Scatter and density plots
def plotScatterMatrix(df, plotSize, textSize):

    """User Defined function to create Scatterpolt Matrix"""


    df = df.select_dtypes(include =[np.number]) # keep only numerical columns
    # Remove rows and columns that would lead to df being singular
    df = df.dropna('columns')
    df = df[[col for col in df if df[col].nunique() > 1]] # keep columns where there are more than 1 unique values
    columnNames = list(df)
    if len(columnNames) > 10: # reduce the number of columns for matrix inversion of kernel density plots
        columnNames = columnNames[:10]
    df = df[columnNames]
    ax = pd.plotting.scatter_matrix(df, alpha=0.75, figsize=[plotSize, plotSize], diagonal='kde')
    corrs = df.corr().values
    for i, j in zip(*plt.np.triu_indices_from(ax, k = 1)):
        ax[i, j].annotate('Corr. coef = %.3f' % corrs[i, j], (0.8, 0.2), xycoords='axes fraction', ha='center', va='center', size=textSize)
    plt.suptitle('Scatter and Density Plot')
    plt.show()
In [46]:
print(plotScatterMatrix.__doc__)
User Defined function to create Scatterpolt Matrix

Exploring Test Annotation file

Loading TestAnnotation.csv to explore it

In [47]:
nRowsRead = None
# anno_test.csv has 8041 rows in reality, but we are only loading/previewing the first 1000 rows
test_anno_df = pd.read_csv('/content/drive/MyDrive/CapstoneProject/Annotations/TestAnnotation.csv')
test_anno_df.rename(columns={'Bounding Box coordinates': 'x0', 'Unnamed: 2': 'y0','Unnamed: 3': 'x1','Unnamed: 4': 'y1'}, inplace=True)
test_anno_df.dataframeName = 'TestAnnotation.csv'
nRow, nCol = test_anno_df.shape
print(f'There are {nRow} rows and {nCol} columns')
There are 8041 rows and 6 columns
In [48]:
test_anno_df.head()
Out[48]:
Image Name x0 y0 x1 y1 Image class
0 00001.jpg 30 52 246 147 181
1 00002.jpg 100 19 576 203 103
2 00003.jpg 51 105 968 659 145
3 00004.jpg 67 84 581 407 187
4 00005.jpg 140 151 593 339 185

Checking Test Annotation files datatypes, there is no issue with the datatypes and every column is mapped properly

In [49]:
test_anno_df.info()
<class 'pandas.core.frame.DataFrame'>
RangeIndex: 8041 entries, 0 to 8040
Data columns (total 6 columns):
 #   Column       Non-Null Count  Dtype 
---  ------       --------------  ----- 
 0   Image Name   8041 non-null   object
 1   x0           8041 non-null   int64 
 2   y0           8041 non-null   int64 
 3   x1           8041 non-null   int64 
 4   y1           8041 non-null   int64 
 5   Image class  8041 non-null   int64 
dtypes: int64(5), object(1)
memory usage: 377.0+ KB

Counting Unique Classes in Test Annotations file

In [54]:
# counting unique values
n = len(pd.unique(test_anno_df['Image class']))
print(n)
196

TestAnnotation file has around 8041 rows and with 6 columns like Image Name, x0, y0, x1, Y1 and Image Class. all 8041 images has been mapped to 196 different classes. There is no issue with the images

In [9]:
plotCorrelationMatrix(test_anno_df, 8)
/usr/local/lib/python3.7/dist-packages/ipykernel_launcher.py:7: FutureWarning: In a future version of pandas all arguments of DataFrame.dropna will be keyword-only
  import sys
In [10]:
plotScatterMatrix(test_anno_df, 15, 10)
/usr/local/lib/python3.7/dist-packages/ipykernel_launcher.py:5: FutureWarning: In a future version of pandas all arguments of DataFrame.dropna will be keyword-only
  """

Exploring Train Annotation file

In [4]:
nRowsRead = None
# anno_test.csv has 8041 rows in reality, but we are only loading/previewing the first 1000 rows
train_anno_df = pd.read_csv('/content/drive/MyDrive/CapstoneProject/Annotations/TrainAnnotations.csv')
train_anno_df.rename(columns={'Bounding Box coordinates': 'x0', 'Unnamed: 2': 'y0','Unnamed: 3': 'x1','Unnamed: 4': 'y1'}, inplace=True)
train_anno_df.dataframeName = 'TrainAnnotation.csv'
nRow, nCol = train_anno_df.shape
print(f'There are {nRow} rows and {nCol} columns')
There are 8144 rows and 6 columns
In [13]:
train_anno_df.head()
Out[13]:
Image Name x0 y0 x1 y1 Image class
0 00001.jpg 39 116 569 375 14
1 00002.jpg 36 116 868 587 3
2 00003.jpg 85 109 601 381 91
3 00004.jpg 621 393 1484 1096 134
4 00005.jpg 14 36 133 99 106

Checking Train Annotation files datatypes, there is no issue with the datatypes and every column is mapped properly

In [17]:
train_anno_df.info()
<class 'pandas.core.frame.DataFrame'>
RangeIndex: 8144 entries, 0 to 8143
Data columns (total 6 columns):
 #   Column       Non-Null Count  Dtype 
---  ------       --------------  ----- 
 0   Image Name   8144 non-null   object
 1   x0           8144 non-null   int64 
 2   y0           8144 non-null   int64 
 3   x1           8144 non-null   int64 
 4   y1           8144 non-null   int64 
 5   Image class  8144 non-null   int64 
dtypes: int64(5), object(1)
memory usage: 381.9+ KB

Counting Unique Classes in Train Annotations file

In [55]:
# counting unique values
n = len(pd.unique(train_anno_df['Image class']))
print(n)
196

TrainAnnotation file has around 8041 rows and with 6 columns like Image Name, x0, y0, x1, Y1 and Image Class. all 8041 images has been mapped to 196 different classes. There is no issue with the images

In [14]:
plotCorrelationMatrix(train_anno_df, 8)
/usr/local/lib/python3.7/dist-packages/ipykernel_launcher.py:7: FutureWarning: In a future version of pandas all arguments of DataFrame.dropna will be keyword-only
  import sys
In [15]:
plotScatterMatrix(train_anno_df, 15, 10)
/usr/local/lib/python3.7/dist-packages/ipykernel_launcher.py:5: FutureWarning: In a future version of pandas all arguments of DataFrame.dropna will be keyword-only
  """

Exploring CarNames file

Loading TrainAnnotation.csv to explore it

In [56]:
nRowsRead = None
#Loading name of cars 
carNameDF= pd.read_csv("/content/drive/MyDrive/CapstoneProject/CarNames.csv",header=None)
carNameDF.dataframeName = 'CarNames.csv'
nRow, nCol = carNameDF.shape
print(f'There are {nRow} rows and {nCol} columns')

carNameDF.rename(columns={0: 'CarNames'}, inplace=True)
carNameDF.info()
There are 196 rows and 1 columns
<class 'pandas.core.frame.DataFrame'>
RangeIndex: 196 entries, 0 to 195
Data columns (total 1 columns):
 #   Column    Non-Null Count  Dtype 
---  ------    --------------  ----- 
 0   CarNames  196 non-null    object
dtypes: object(1)
memory usage: 1.7+ KB
In [57]:
carNameDF.head()
Out[57]:
CarNames
0 AM General Hummer SUV 2000
1 Acura RL Sedan 2012
2 Acura TL Sedan 2012
3 Acura TL Type-S 2008
4 Acura TSX Sedan 2012

Getting unqiue count

In [58]:
n = len(pd.unique(carNameDF['CarNames']))
print(n)
196

Totally data has more than 196 unique values

Map training images to its classes

User defined function to create image dataframe

In [19]:
import os
import tensorflow
from keras.preprocessing import image
#from tqdm import tqdm
def createImageDF(imageSourceFolder,target_size_tuple,colormode,totalImagePerClassCap):
    dfRows=[]
    #l = glob.glob('/content/drive/MyDrive/CapstoneProject/CarImages/TrainImages/*/*')
    
    for dirname,_, filenames in os.walk(imageSourceFolder):
            limit=0
            if len(filenames) > totalImagePerClassCap or totalImagePerClassCap == 0:
                limit=len(filenames)
            else:
                limit=totalImagePerClassCap
            for i in range(0,limit):
                    processedRows=[]
                    filename=filenames[i]
                    processedRows.append(filename)
                    pathSplits=dirname.split('/')
                    print(os.path.join(dirname, filename))
                    imgPath=os.path.join(dirname, filename)
                    print(imgPath)
                    img = image.load_img(imgPath)
                    if target_size_tuple!='':
                        img_scaled = image.load_img(imgPath, target_size=target_size_tuple, color_mode = colormode)
                        image_width_scale_fact=target_size_tuple[0]/img.width
                        image_height_scale_fact=target_size_tuple[1]/img.height
                        imgArray = image.img_to_array(img_scaled)
                    else:
                        img
                        image_width_scale_fact=img.width
                        image_height_scale_fact=img.height
                        #img = Image.open(imgPath)
                        imgArray = image.img_to_array(img)
                    #imgArray=np.asarray(img)
                    processedRows.append(imgArray)
                    processedRows.append(pathSplits[len(pathSplits)-1])
                    processedRows.append(image_width_scale_fact)
                    processedRows.append(image_height_scale_fact)
                    dfRows.append(processedRows)
    return pd.DataFrame(dfRows, columns=["Image Name","image_array","model","image_width_scale_fact","image_height_scale_fact"])

User Defined function to load the single score image

In [20]:
def loadSingleScoringImage(imageSourcePath,target_size_tuple,colormode):
    dfRows=[]
    processedRow=[]
    imgPath=imageSourcePath
    print(imgPath)
    img = image.load_img(imgPath)
    pathSplits=imgPath.split('/')
    if target_size_tuple!='':
        img_scaled = image.load_img(imgPath, target_size=target_size_tuple, color_mode = colormode)
        image_width_scale_fact=target_size_tuple[0]/img.width
        image_height_scale_fact=target_size_tuple[1]/img.height
        imgArray = image.img_to_array(img_scaled)
    else:
        img
        image_width_scale_fact=img.width
        image_height_scale_fact=img.height
        #img = Image.open(imgPath)
        imgArray = image.img_to_array(img)
    #imgArray=np.asarray(img)
    processedRow.append(pathSplits[len(pathSplits)-1])
    processedRow.append(imgArray)
    processedRow.append(pathSplits[len(pathSplits)-2])
    processedRow.append(image_width_scale_fact)
    processedRow.append(image_height_scale_fact)
    dfRows.append(processedRow)
    return pd.DataFrame(dfRows, columns=["Image Name","image_array","model","image_width_scale_fact","image_height_scale_fact"])
In [22]:
# import glob
# TrainImage = glob.glob('/content/drive/MyDrive/CapstoneProject/CarImages/TrainImages/*')

# FirstTime = 0
# for ImgFolder in TrainImage:
#   try:
#     #print(ImgFolder)
#     trainImageDF=createImageDF(ImgFolder,'',"rgb",5)
#     if FirstTime == 0:
#       mergedTrainImageDF = trainImageDF
#       FirstTime = FirstTime +  1
#     else:
#       mergedTrainImageDF = mergedTrainImageDF.append(trainImageDF)
#   except:
#     print("Success")

#As GPU system got full, I have loaded the sample classes and created Image dataframe
trainImageDF=createImageDF('/content/drive/MyDrive/CapstoneProject/CarImages/TrainImages/Audi 100 Sedan 1994','',"rgb",2)
/content/drive/MyDrive/CapstoneProject/CarImages/TrainImages/Audi 100 Sedan 1994/07076.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TrainImages/Audi 100 Sedan 1994/07076.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TrainImages/Audi 100 Sedan 1994/06568.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TrainImages/Audi 100 Sedan 1994/06568.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TrainImages/Audi 100 Sedan 1994/01502.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TrainImages/Audi 100 Sedan 1994/01502.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TrainImages/Audi 100 Sedan 1994/07693.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TrainImages/Audi 100 Sedan 1994/07693.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TrainImages/Audi 100 Sedan 1994/03717.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TrainImages/Audi 100 Sedan 1994/03717.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TrainImages/Audi 100 Sedan 1994/04639.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TrainImages/Audi 100 Sedan 1994/04639.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TrainImages/Audi 100 Sedan 1994/05326.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TrainImages/Audi 100 Sedan 1994/05326.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TrainImages/Audi 100 Sedan 1994/00865.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TrainImages/Audi 100 Sedan 1994/00865.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TrainImages/Audi 100 Sedan 1994/00266.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TrainImages/Audi 100 Sedan 1994/00266.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TrainImages/Audi 100 Sedan 1994/02350.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TrainImages/Audi 100 Sedan 1994/02350.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TrainImages/Audi 100 Sedan 1994/06947.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TrainImages/Audi 100 Sedan 1994/06947.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TrainImages/Audi 100 Sedan 1994/08043.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TrainImages/Audi 100 Sedan 1994/08043.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TrainImages/Audi 100 Sedan 1994/04380.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TrainImages/Audi 100 Sedan 1994/04380.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TrainImages/Audi 100 Sedan 1994/05594.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TrainImages/Audi 100 Sedan 1994/05594.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TrainImages/Audi 100 Sedan 1994/04758.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TrainImages/Audi 100 Sedan 1994/04758.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TrainImages/Audi 100 Sedan 1994/05181.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TrainImages/Audi 100 Sedan 1994/05181.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TrainImages/Audi 100 Sedan 1994/07774.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TrainImages/Audi 100 Sedan 1994/07774.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TrainImages/Audi 100 Sedan 1994/04949.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TrainImages/Audi 100 Sedan 1994/04949.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TrainImages/Audi 100 Sedan 1994/01072.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TrainImages/Audi 100 Sedan 1994/01072.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TrainImages/Audi 100 Sedan 1994/05464.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TrainImages/Audi 100 Sedan 1994/05464.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TrainImages/Audi 100 Sedan 1994/02770.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TrainImages/Audi 100 Sedan 1994/02770.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TrainImages/Audi 100 Sedan 1994/07566.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TrainImages/Audi 100 Sedan 1994/07566.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TrainImages/Audi 100 Sedan 1994/00402.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TrainImages/Audi 100 Sedan 1994/00402.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TrainImages/Audi 100 Sedan 1994/03985.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TrainImages/Audi 100 Sedan 1994/03985.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TrainImages/Audi 100 Sedan 1994/03253.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TrainImages/Audi 100 Sedan 1994/03253.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TrainImages/Audi 100 Sedan 1994/02600.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TrainImages/Audi 100 Sedan 1994/02600.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TrainImages/Audi 100 Sedan 1994/08137.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TrainImages/Audi 100 Sedan 1994/08137.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TrainImages/Audi 100 Sedan 1994/00066.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TrainImages/Audi 100 Sedan 1994/00066.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TrainImages/Audi 100 Sedan 1994/06224.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TrainImages/Audi 100 Sedan 1994/06224.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TrainImages/Audi 100 Sedan 1994/04447.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TrainImages/Audi 100 Sedan 1994/04447.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TrainImages/Audi 100 Sedan 1994/05177.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TrainImages/Audi 100 Sedan 1994/05177.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TrainImages/Audi 100 Sedan 1994/03533.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TrainImages/Audi 100 Sedan 1994/03533.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TrainImages/Audi 100 Sedan 1994/01782.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TrainImages/Audi 100 Sedan 1994/01782.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TrainImages/Audi 100 Sedan 1994/08144.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TrainImages/Audi 100 Sedan 1994/08144.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TrainImages/Audi 100 Sedan 1994/04549.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TrainImages/Audi 100 Sedan 1994/04549.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TrainImages/Audi 100 Sedan 1994/08010.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TrainImages/Audi 100 Sedan 1994/08010.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TrainImages/Audi 100 Sedan 1994/03765.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TrainImages/Audi 100 Sedan 1994/03765.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TrainImages/Audi 100 Sedan 1994/02176.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TrainImages/Audi 100 Sedan 1994/02176.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TrainImages/Audi 100 Sedan 1994/05717.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TrainImages/Audi 100 Sedan 1994/05717.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TrainImages/Audi 100 Sedan 1994/02887.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TrainImages/Audi 100 Sedan 1994/02887.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TrainImages/Audi 100 Sedan 1994/05968.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TrainImages/Audi 100 Sedan 1994/05968.jpg
In [23]:
trainImageDF.shape
Out[23]:
(41, 5)

Merging TrainImageDF and Train Annotations

In [24]:
trainImageDF.head()
Out[24]:
Image Name image_array model image_width_scale_fact image_height_scale_fact
0 07076.jpg [[[44.0, 32.0, 8.0], [83.0, 68.0, 45.0], [70.0... Audi 100 Sedan 1994 128 96
1 06568.jpg [[[112.0, 110.0, 115.0], [191.0, 189.0, 194.0]... Audi 100 Sedan 1994 128 96
2 01502.jpg [[[248.0, 255.0, 245.0], [246.0, 254.0, 243.0]... Audi 100 Sedan 1994 160 80
3 07693.jpg [[[61.0, 61.0, 61.0], [85.0, 85.0, 85.0], [64.... Audi 100 Sedan 1994 600 297
4 03717.jpg [[[163.0, 173.0, 182.0], [166.0, 176.0, 185.0]... Audi 100 Sedan 1994 150 113
In [25]:
train_anno_df.head()
Out[25]:
Image Name x0 y0 x1 y1 Image class
0 00001.jpg 39 116 569 375 14
1 00002.jpg 36 116 868 587 3
2 00003.jpg 85 109 601 381 91
3 00004.jpg 621 393 1484 1096 134
4 00005.jpg 14 36 133 99 106

Merging the trainImageDF and train_anno_df using the common filed - Image Name

Merged Train Image and Train Annonation and Mapped Image classes

In [26]:
# Lodaing annotation details for images from train_anno_df i.e. train annotations
trainMergedImageDF=pd.merge(trainImageDF,train_anno_df, on=['Image Name'])
In [27]:
trainMergedImageDF.head()
Out[27]:
Image Name image_array model image_width_scale_fact image_height_scale_fact x0 y0 x1 y1 Image class
0 07076.jpg [[[44.0, 32.0, 8.0], [83.0, 68.0, 45.0], [70.0... Audi 100 Sedan 1994 128 96 2 34 128 81 17
1 06568.jpg [[[112.0, 110.0, 115.0], [191.0, 189.0, 194.0]... Audi 100 Sedan 1994 128 96 1 43 128 91 17
2 01502.jpg [[[248.0, 255.0, 245.0], [246.0, 254.0, 243.0]... Audi 100 Sedan 1994 160 80 6 3 158 77 17
3 07693.jpg [[[61.0, 61.0, 61.0], [85.0, 85.0, 85.0], [64.... Audi 100 Sedan 1994 600 297 21 57 554 278 17
4 03717.jpg [[[163.0, 173.0, 182.0], [166.0, 176.0, 185.0]... Audi 100 Sedan 1994 150 113 9 25 136 92 17

Successfully Mapped image classes now, the trainMergedImageDF dataframe has complete information to generate bounding box and for developing model

Function to create bounding box on image

In [37]:
import matplotlib.pyplot as plt
import matplotlib.patches as patches

def drawBoxedImages(dataFrame,featureCol,x0,y0,x1,y1,boxlinewidth,boxedgecolor):
  fig,ax=plt.subplots(len(dataFrame))
  fig.set_size_inches(30,30)
  # Display the image
  for index, row in dataFrame.iterrows():
    data=row[featureCol]
    img = image.array_to_img(data)
    ax[index].imshow(img)
    image_height, image_width, _ = data.shape
    rect=[]
    rect.append(patches.Rectangle((row[x0], row[y0]), row[x1] - row[x0], row[y1] - row[y0], linewidth=boxlinewidth, edgecolor=boxedgecolor, facecolor='none'))
    # Add the patch to the Axes
    for l in range(0,len(rect)):
      ax[index].add_patch(rect[l])
  plt.show()
In [38]:
def drawImages(dataFrame,featureCol):
  fig,ax=plt.subplots(len(dataFrame))
  fig.set_size_inches(30,30)
  # Display the image
  for index, row in dataFrame.iterrows():
    data=row[featureCol]
    img = image.array_to_img(data)
    ax[index].imshow(img)
  plt.show()

Generating Bounding Box Images for Train data

Calling drawBoxedImages function to generate sample bounding box images for training set

In [39]:
drawBoxedImages(trainMergedImageDF.head(5),'image_array','x0','y0','x1','y1',2,'r')

Map testing images to its classes

In [21]:
testImageDF=createImageDF('/content/drive/MyDrive/CapstoneProject/CarImages/TestImages/Audi 100 Sedan 1994','',"rgb",2)
/content/drive/MyDrive/CapstoneProject/CarImages/TestImages/Audi 100 Sedan 1994/04287.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TestImages/Audi 100 Sedan 1994/04287.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TestImages/Audi 100 Sedan 1994/00082.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TestImages/Audi 100 Sedan 1994/00082.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TestImages/Audi 100 Sedan 1994/01335.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TestImages/Audi 100 Sedan 1994/01335.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TestImages/Audi 100 Sedan 1994/01012.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TestImages/Audi 100 Sedan 1994/01012.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TestImages/Audi 100 Sedan 1994/04561.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TestImages/Audi 100 Sedan 1994/04561.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TestImages/Audi 100 Sedan 1994/00573.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TestImages/Audi 100 Sedan 1994/00573.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TestImages/Audi 100 Sedan 1994/04250.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TestImages/Audi 100 Sedan 1994/04250.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TestImages/Audi 100 Sedan 1994/01917.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TestImages/Audi 100 Sedan 1994/01917.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TestImages/Audi 100 Sedan 1994/02143.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TestImages/Audi 100 Sedan 1994/02143.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TestImages/Audi 100 Sedan 1994/01617.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TestImages/Audi 100 Sedan 1994/01617.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TestImages/Audi 100 Sedan 1994/02187.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TestImages/Audi 100 Sedan 1994/02187.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TestImages/Audi 100 Sedan 1994/00789.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TestImages/Audi 100 Sedan 1994/00789.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TestImages/Audi 100 Sedan 1994/00333.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TestImages/Audi 100 Sedan 1994/00333.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TestImages/Audi 100 Sedan 1994/04209.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TestImages/Audi 100 Sedan 1994/04209.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TestImages/Audi 100 Sedan 1994/00949.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TestImages/Audi 100 Sedan 1994/00949.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TestImages/Audi 100 Sedan 1994/03791.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TestImages/Audi 100 Sedan 1994/03791.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TestImages/Audi 100 Sedan 1994/04603.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TestImages/Audi 100 Sedan 1994/04603.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TestImages/Audi 100 Sedan 1994/01893.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TestImages/Audi 100 Sedan 1994/01893.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TestImages/Audi 100 Sedan 1994/02214.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TestImages/Audi 100 Sedan 1994/02214.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TestImages/Audi 100 Sedan 1994/00891.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TestImages/Audi 100 Sedan 1994/00891.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TestImages/Audi 100 Sedan 1994/05077.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TestImages/Audi 100 Sedan 1994/05077.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TestImages/Audi 100 Sedan 1994/05891.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TestImages/Audi 100 Sedan 1994/05891.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TestImages/Audi 100 Sedan 1994/07381.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TestImages/Audi 100 Sedan 1994/07381.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TestImages/Audi 100 Sedan 1994/05935.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TestImages/Audi 100 Sedan 1994/05935.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TestImages/Audi 100 Sedan 1994/06204.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TestImages/Audi 100 Sedan 1994/06204.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TestImages/Audi 100 Sedan 1994/05969.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TestImages/Audi 100 Sedan 1994/05969.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TestImages/Audi 100 Sedan 1994/05246.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TestImages/Audi 100 Sedan 1994/05246.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TestImages/Audi 100 Sedan 1994/04860.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TestImages/Audi 100 Sedan 1994/04860.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TestImages/Audi 100 Sedan 1994/07284.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TestImages/Audi 100 Sedan 1994/07284.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TestImages/Audi 100 Sedan 1994/06963.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TestImages/Audi 100 Sedan 1994/06963.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TestImages/Audi 100 Sedan 1994/06767.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TestImages/Audi 100 Sedan 1994/06767.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TestImages/Audi 100 Sedan 1994/06283.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TestImages/Audi 100 Sedan 1994/06283.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TestImages/Audi 100 Sedan 1994/05319.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TestImages/Audi 100 Sedan 1994/05319.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TestImages/Audi 100 Sedan 1994/05010.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TestImages/Audi 100 Sedan 1994/05010.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TestImages/Audi 100 Sedan 1994/04644.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TestImages/Audi 100 Sedan 1994/04644.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TestImages/Audi 100 Sedan 1994/05355.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TestImages/Audi 100 Sedan 1994/05355.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TestImages/Audi 100 Sedan 1994/05455.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TestImages/Audi 100 Sedan 1994/05455.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TestImages/Audi 100 Sedan 1994/07491.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TestImages/Audi 100 Sedan 1994/07491.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TestImages/Audi 100 Sedan 1994/08039.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TestImages/Audi 100 Sedan 1994/08039.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TestImages/Audi 100 Sedan 1994/04926.jpg
/content/drive/MyDrive/CapstoneProject/CarImages/TestImages/Audi 100 Sedan 1994/04926.jpg
In [32]:
testImageDF.shape
Out[32]:
(40, 5)

Merging the testImageDF and test_anno_df using the common filed - Image Name

In [31]:
testImageDF.head()
Out[31]:
Image Name image_array model image_width_scale_fact image_height_scale_fact
0 04287.jpg [[[118.0, 107.0, 77.0], [122.0, 111.0, 81.0], ... Audi 100 Sedan 1994 640 480
1 00082.jpg [[[153.0, 154.0, 156.0], [152.0, 153.0, 155.0]... Audi 100 Sedan 1994 160 120
2 01335.jpg [[[117.0, 143.0, 144.0], [72.0, 93.0, 96.0], [... Audi 100 Sedan 1994 320 240
3 01012.jpg [[[110.0, 110.0, 118.0], [83.0, 86.0, 93.0], [... Audi 100 Sedan 1994 448 336
4 04561.jpg [[[253.0, 253.0, 253.0], [253.0, 253.0, 253.0]... Audi 100 Sedan 1994 150 100
In [33]:
test_anno_df.head()
Out[33]:
Image Name x0 y0 x1 y1 Image class
0 00001.jpg 30 52 246 147 181
1 00002.jpg 100 19 576 203 103
2 00003.jpg 51 105 968 659 145
3 00004.jpg 67 84 581 407 187
4 00005.jpg 140 151 593 339 185
In [34]:
# Lodaing annotation details for images from train_anno_df i.e. train annotations
testMergedImageDF=pd.merge(testImageDF,test_anno_df, on=['Image Name'])

Merged Test Image and Test Annonation and Mapped Image classes

In [35]:
testMergedImageDF.head(10)
Out[35]:
Image Name image_array model image_width_scale_fact image_height_scale_fact x0 y0 x1 y1 Image class
0 04287.jpg [[[118.0, 107.0, 77.0], [122.0, 111.0, 81.0], ... Audi 100 Sedan 1994 640 480 63 50 581 348 17
1 00082.jpg [[[153.0, 154.0, 156.0], [152.0, 153.0, 155.0]... Audi 100 Sedan 1994 160 120 16 14 145 106 17
2 01335.jpg [[[117.0, 143.0, 144.0], [72.0, 93.0, 96.0], [... Audi 100 Sedan 1994 320 240 14 34 279 183 17
3 01012.jpg [[[110.0, 110.0, 118.0], [83.0, 86.0, 93.0], [... Audi 100 Sedan 1994 448 336 64 101 323 294 17
4 04561.jpg [[[253.0, 253.0, 253.0], [253.0, 253.0, 253.0]... Audi 100 Sedan 1994 150 100 6 32 140 100 17
5 00573.jpg [[[125.0, 121.0, 122.0], [144.0, 140.0, 141.0]... Audi 100 Sedan 1994 240 180 9 44 230 134 17
6 04250.jpg [[[251.0, 248.0, 241.0], [251.0, 248.0, 241.0]... Audi 100 Sedan 1994 584 328 7 107 575 301 17
7 01917.jpg [[[72.0, 72.0, 72.0], [46.0, 46.0, 46.0], [49.... Audi 100 Sedan 1994 142 94 5 22 142 84 17
8 02143.jpg [[[56.0, 30.0, 39.0], [47.0, 11.0, 21.0], [64.... Audi 100 Sedan 1994 800 495 13 26 796 479 17
9 01617.jpg [[[88.0, 106.0, 48.0], [145.0, 163.0, 105.0], ... Audi 100 Sedan 1994 259 194 32 59 200 165 17

Generating Bounding Box Images for Test data

In [40]:
drawBoxedImages(testMergedImageDF.head(10),'image_array','x0','y0','x1','y1',2,'r')

Building Base Model [ Resnet - PyTorch Framework]

Loading required libraries for modelling

In [ ]:
import matplotlib.pyplot as plt
import numpy as np

import torch
import torch.nn as nn
import torch.optim as optim
import torchvision
import torchvision.models as models
import torchvision.transforms as transforms

import time
import os
import PIL.Image as Image
from IPython.display import display

device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")
print(device)
print(torch.cuda.get_device_name(device))
cuda:0
Tesla T4

Transforming the images by performing multiple operations like Resizing, Horizontal Flip, Rotations on Train and Test data

In [ ]:
dataset_dir = "/content/drive/MyDrive/CapstoneProject/CarImages/"

train_tfms = transforms.Compose([transforms.Resize((400, 400)),
                                 transforms.RandomHorizontalFlip(),
                                 transforms.RandomRotation(15),
                                 transforms.ToTensor(),
                                 transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))])
test_tfms = transforms.Compose([transforms.Resize((400, 400)),
                                transforms.ToTensor(),
                                transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))])

dataset = torchvision.datasets.ImageFolder(root=dataset_dir+"train", transform = train_tfms)
trainloader = torch.utils.data.DataLoader(dataset, batch_size = 32, shuffle=True, num_workers = 2)

dataset2 = torchvision.datasets.ImageFolder(root=dataset_dir+"test", transform = test_tfms)
testloader = torch.utils.data.DataLoader(dataset2, batch_size = 32, shuffle=False, num_workers = 2)

Creating the user defined function for train_model

In [ ]:
def train_model(model, criterion, optimizer, scheduler, n_epochs = 5):
    
    losses = []
    accuracies = []
    test_accuracies = []
    # set the model to train mode initially
    model.train()
    for epoch in range(n_epochs):
        since = time.time()
        running_loss = 0.0
        running_correct = 0.0
        for i, data in enumerate(trainloader, 0):

            # get the inputs and assign them to cuda
            inputs, labels = data
            #inputs = inputs.to(device).half() # uncomment for half precision model
            inputs = inputs.to(device)
            labels = labels.to(device)
            optimizer.zero_grad()
            
            # forward + backward + optimize
            outputs = model(inputs)
            _, predicted = torch.max(outputs.data, 1)
            loss = criterion(outputs, labels)
            loss.backward()
            optimizer.step()
            
            # calculate the loss/acc later
            running_loss += loss.item()
            running_correct += (labels==predicted).sum().item()

        epoch_duration = time.time()-since
        epoch_loss = running_loss/len(trainloader)
        epoch_acc = 100/32*running_correct/len(trainloader)
        print("Epoch %s, duration: %d s, loss: %.4f, acc: %.4f" % (epoch+1, epoch_duration, epoch_loss, epoch_acc))
        
        losses.append(epoch_loss)
        accuracies.append(epoch_acc)
        
        # switch the model to eval mode to evaluate on test data
        model.eval()
        test_acc = eval_model(model)
        test_accuracies.append(test_acc)
        
        # re-set the model to train mode after validating
        model.train()
        scheduler.step(test_acc)
        since = time.time()
    print('Finished Training')
    return model, losses, accuracies, test_accuracies
In [ ]:
def eval_model(model):
    correct = 0.0
    total = 0.0
    with torch.no_grad():
        for i, data in enumerate(testloader, 0):
            images, labels = data
            #images = images.to(device).half() # uncomment for half precision model
            images = images.to(device)
            labels = labels.to(device)
            
            outputs = model_ft(images)
            _, predicted = torch.max(outputs.data, 1)
            
            total += labels.size(0)
            correct += (predicted == labels).sum().item()

    test_acc = 100.0 * correct / total
    print('Accuracy of the network on the test images: %d %%' % (
        test_acc))
    return test_acc

Loading Resnet Pretrained model and using it in PyTorch Framework

In [ ]:
model_ft = models.resnet34(pretrained=True)
num_ftrs = model_ft.fc.in_features


model_ft.fc = nn.Linear(num_ftrs, 196)
model_ft = model_ft.to(device)


criterion = nn.CrossEntropyLoss()
optimizer = optim.SGD(model_ft.parameters(), lr=0.01, momentum=0.9)

lrscheduler = optim.lr_scheduler.ReduceLROnPlateau(optimizer, mode='max', patience=3, threshold = 0.9)
Downloading: "https://download.pytorch.org/models/resnet34-b627a593.pth" to /root/.cache/torch/hub/checkpoints/resnet34-b627a593.pth

Models Keys Settings

  1. Model Fitting - Resnet pretrained model
  2. Criterion - CrossEntropyLoss
  3. Optimizer - SGD Optimizer
  4. lrscheduler - ReduceLROnPlateau
  5. Epoch - 10
In [ ]:
model_ft, training_losses, training_accs, test_accs = train_model(model_ft, criterion, optimizer, lrscheduler, n_epochs=10)
Epoch 1, duration: 995 s, loss: 3.9668, acc: 16.9485
Accuracy of the network on the test images: 34 %
Epoch 2, duration: 151 s, loss: 1.6283, acc: 57.6348
Accuracy of the network on the test images: 57 %
Epoch 3, duration: 154 s, loss: 0.8562, acc: 76.7770
Accuracy of the network on the test images: 66 %
Epoch 4, duration: 154 s, loss: 0.5250, acc: 85.2206
Accuracy of the network on the test images: 74 %
Epoch 5, duration: 152 s, loss: 0.3487, acc: 90.1593
Accuracy of the network on the test images: 80 %
Epoch 6, duration: 154 s, loss: 0.2317, acc: 93.5172
Accuracy of the network on the test images: 82 %
Epoch 7, duration: 155 s, loss: 0.1709, acc: 95.6250
Accuracy of the network on the test images: 82 %
Epoch 8, duration: 153 s, loss: 0.0799, acc: 98.1618
Accuracy of the network on the test images: 89 %
Epoch 9, duration: 154 s, loss: 0.0548, acc: 98.7868
Accuracy of the network on the test images: 90 %
Epoch 10, duration: 154 s, loss: 0.0476, acc: 99.0196
Accuracy of the network on the test images: 90 %
Finished Training

Plotting Training, Testing Accuracy and Training Loss

In [ ]:
# plot the stats

f, axarr = plt.subplots(2,2, figsize = (12, 8))
axarr[0, 0].plot(training_losses)
axarr[0, 0].set_title("Training loss")
axarr[0, 1].plot(training_accs)
axarr[0, 1].set_title("Training acc")
axarr[1, 0].plot(test_accs)

axarr[1, 0].set_title("Test acc")
Out[ ]:
Text(0.5, 1.0, 'Test acc')
In [ ]:
# tie the class indices to their names

def find_classes(dir):
    classes = os.listdir(dir)
    classes.sort()
    class_to_idx = {classes[i]: i for i in range(len(classes))}
    return classes, class_to_idx
classes, c_to_idx = find_classes(dataset_dir+"train")

Testing the base model

In [ ]:
# test the model on random images
dataset_dir = "/content/drive/MyDrive/CapstoneProject/CarImages"
torch.save(model_ft,"/content/drive/MyDrive/CapstoneProject/CarImages/working/model.h5")

# switch the model to evaluation mode to make dropout and batch norm work in eval mode
model_ft.eval()

# transforms for the input image
loader = transforms.Compose([transforms.Resize((400, 400)),
                                transforms.ToTensor(),
                                transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))])
image = Image.open(dataset_dir+"test/Mercedes-Benz C-Class Sedan 2012/01977.jpg")
image = loader(image).float()
image = torch.autograd.Variable(image, requires_grad=True)
image = image.unsqueeze(0)
image = image.cuda()

output = model_ft(image)
conf, predicted = torch.max(output.data, 1)
In [ ]:
# get the class name of the prediction
display(Image.open(dataset_dir+"test/Mercedes-Benz C-Class Sedan 2012/01977.jpg"))
print(classes[predicted.item()], "confidence: ", conf.item())
Mercedes-Benz C-Class Sedan 2012 confidence:  15.01318073272705
In [ ]:
dataset_dir = "/content/drive/MyDrive/CapstoneProject/CarImages/test"
In [ ]:
import os
listFolders = os.listdir(dataset_dir)
In [ ]:
import glob

classNames = []
filePaths=[]
for folder in listFolders:
    i=0
    for file in glob.glob(dataset_dir+"/"+folder+"/*.jpg"):
        if(i==5):
            break
        classNames.append(folder)
        filePaths.append(file)
        i=i+1

Loading the test image and its prediction

In [ ]:
count = 0
for imgPath in filePaths:
# transforms for the input image
    try:
        loader = transforms.Compose([transforms.Resize((400, 400)),
                                    transforms.ToTensor(),
                                    transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))])
        image = Image.open(imgPath)
        if(image is None):
            continue
        image = loader(image).float()
        image = torch.autograd.Variable(image, requires_grad=True)
        image = image.unsqueeze(0)
        image = image.cuda()
        output = model_ft(image)
        conf, predicted = torch.max(output.data, 1)
        display(Image.open(imgPath))
        print(classes[predicted.item()], "confidence: ", conf.item())
        count += 1
        if count >= 10:
            break
    except:
        print("Error")
Aston Martin Virage Convertible 2012 confidence:  15.837179183959961
Aston Martin V8 Vantage Convertible 2012 confidence:  10.637139320373535
Aston Martin Virage Convertible 2012 confidence:  18.681026458740234
Aston Martin Virage Convertible 2012 confidence:  16.215911865234375
Aston Martin Virage Convertible 2012 confidence:  16.623130798339844
Acura TL Type-S 2008 confidence:  12.212014198303223
Acura TL Type-S 2008 confidence:  17.66625213623047
Acura TL Type-S 2008 confidence:  18.62812614440918
Acura TL Type-S 2008 confidence:  18.368614196777344
Mercedes-Benz SL-Class Coupe 2009 confidence:  10.162840843200684
In [ ]: